Purpose of Changes and their Description

Why "Weak"?

@jmdkastro formulated a well-defined theory of merit-based sortition plus simulation. This rhymes with that, but is not exactly that.

A consequence of #567 will be way greater stagnation in the active set, as those with scores have an actual opportunity to make it into their topic's epochs. Hence, there is a need to quickly add a non-corrosive form of permeability to the network. This route maximizes our ability to serve large cohorts of workers while enabling for more sophisticated methods later, potentially as fast-follows.

How it Works

Definition

• Active set = Your inference/forecast/reputer loss payload is considered as part of rewards/inference synthesis calculations
• Passive set = Your inference/forecast/reputer loss payload is not considered as part of rewards/inference synthesis calculations

We differentiate between these 2 sets because calculations for the active set are expensive => this must be bounded. However, we must cycle through participants, so we now need a name for those "passively considered" actors.

When scores are calculated in rewards

• An EMA score is updated for each actor of each type
• A topic-defined quantile of the active set is taken for each actor type (inferer, forecaster, reputer)
  • This will be used in the next epoch
  • For example if EMA score among active set is [10,20,30] then topic quantile might be 12 (linear interpolation)

In the next epoch

• We skim the top in an online fashion i.e.
  • If we have not yet met our limit per actor type per epoch, then we add the actor payload to the active set
  • If we have met our limit per actor, then we ask if the EMA score of the new actor is better than the lowest in the current active set.
    • If yes, then we pop the lowest scorer by EMA score out of the active set and update their EMA using the topic quantile of the previous epoch
    • If no, then we update the EMA score of the rejected actor using the topic quantile of the previous epoch

In summary, if you're in the active set, your EMA score gets updated based on your performance in this round, and if you're int he passive set, your EMA score gets updated based on the performance of the previous epoch's active set. In either case, we do calculations in an online fashion => the true limit of actors for which we can do calculations is the cumulative blockspace within a worker submission window.

Link(s) to Ticket(s) or Issue(s) resolved by this PR

Are these changes tested and documented?

• If tested, please describe how. If not, why tests are not needed.
• If documented, please describe where. If not, describe why docs are not needed.
• Added to Unreleased section of CHANGELOG.md?

Still Left Todo

• Fix tests
• Add more coverage